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ABSTRACT 

The convergence of solutions to the discrete or sampled time linear quadratic regulator 
problem and associated Riccati equation for infinite dimensional systems to the solutions 
to the corresponding continuous time problem and equation, as the length of the sampling 
interval (the sampling rate) tends toward zero (infinity) is established. Both the finite and 
infinite time horizon problems are studied. In the finite time horizon case, strong continuity 
of the operators which define the control system and performance index together with a 
stability and consistency condition on the sampling scheme are required. For the infinite 
time horizon problem, in addition, the sampled systems must be stabilizable and detectable, 
uniformly with respect to the sampling rate. Classes of systems for which this condition can 
be verified are discussed. Results of numerical studies involving the control of a heat/ diffusion 
equation, a hereditary of delay system, and a flexible beam are presented and discussed. 
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1. Introduction. In this paper we consider the convergence of closed-loop solu- 
tions to discrete or sampled time linear quadratic (LQ) optimal control problems and 
the associated Riccati equations for infinite dimensional systems defined on Hilbert 
spaces to the solutions to the corresponding continuous time problems and Riccati 
equations, as the length of the sampling interval tends toward zero. With the advent 
and proliferation of micro-computers, and control tasks becoming ever more complex 
(for example, the stabilization of large flexible spacecraft), the roles played by dis- 
crete or sampled time control design techniques and distributed parameter systems 
have become increasingly more important. It has become necessary, therefore, to 
develop extensions of many of the familiar results for finite dimensional systems to 
an infinite dimensional setting. One area that has recently received a great deal of 
attention has been the LQ theory. Certain aspects of the linear-quadratic approach 
to control design for both continuous and sampled time infinite dimensional systems 
have been studied extensively. In particular, these aspects include, for example, the 
linear state feedback structure of the optimal control law, the optimal LQG estimator 
and compensator problems, boundary control, and finite dimensional approximation 
(for specific references, see below). But to the best of our knowledge, however, the 
inter-relation between the continuous and discrete time theories, which in the finite 
dimensional case is well understood, has not as of yet, been looked at in the context 
of infinite dimensional systems. Such a study would be useful, for example, because 
typically in engineering practice, the discrete and continuous time LQ theories are 
applied interchangeably without regard to as to whether or not the actual system is 
continuous or discrete in nature. In particular, due to hardware constraints, most sys- 
tems occurring in engineering practice are in fact discrete. However, if the sampling 
is considered to be rapid enough, the system may be treated as continuous when an 
optimal control law, state estimator, or compensator is designed. Our work is largely 
motivated by the fact that the results we shall present here will serve to, in some 
sense, justify this approach. 

We note that in finite dimensions, where strong and uniform norm convergence of 
linear operators are equivalent, the continuous dependence with respect to sampling of 
the solution to the linear quadratic control problem and associated Riccati equation 
is straight forward. Indeed, in [Le] the continuous time theory is established by 
first deriving the discrete time results, which are fundamentally algebraic in nature, 
and then taking the limit as the length of the sampling interval tends toward zero. 
However, in infinite dimensions, as is typically the case, the desired convergence is 
less obvious. This is especially true in the case of the infinite time horizon problem. 
It is this problem that we address here. 

We consider both the finite and infinite time horizon problems. In the case of 
the finite time horizon problem, under the assumption of strong continuity of the 
operators which define the control system and performance index, together with a 
stability and consistency hypothesis on the sampling scheme, we are able to deduce 
the desired convergence. We must develop an appropriate framework to facilitate the 
comparison of discrete and continuous time operator families. For this purpose we rely 
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heavily upon Kato’s [K] treatment of discrete semigroups. In the case of the infinite 
time horizon problem we must additionally assume stabilizability and detectability of 
the discrete time systems with some degree of uniformity in the sampling rate. The 
notion of stabilizability /detectability uniform with respect to sampling will be made 
precise in Section 3 below. We are able to establish that if the continuous time system 
is stabilizable and detectable via finite rank feedback, and if zero-order hold sampling 
is employed, then the resulting discrete time systems are uniformly stabilizable and 
detectable for sufficiently small sampling interval. We also have a result concerning 
the uniform stabilizability and detectability of parabolic systems. However, this result 
will not be discussed here, but rather in a forthcoming manuscript. 

Our treatment is functional analytic in nature, and is similar in spirit to the many 
recent studies of convergence of solutions to LQ control and estimation problems 
and the associated Riccati equations under state (space) approximation (i.e. finite 
difference, modal, or finite element, for example). See, for example, [BK], [BW], 
[G],[GA], [GR], and [Wj. For the discrete time LQ theory for infinite dimensional 
systems, we rely heavily on the well known results contained in [HH], [LCB], and [Z] . 

In addition to our theoretical results, we have included the results of some of our 
numerical convergence studies. We present and discuss our findings for the infinite 
time horizon LQ optimal control problems for a one dimensional heat or diffusion 
equation, a one dimensional hereditary or delay system, and a hybrid system of or- 
dinary and partial differential equations describing the small amplitude transverse 
vibration of a cantilevered Voigt-Kelvin viscoelastic beam with tip mass. 

An outline of the remainder of the paper is as follows. In section 2 we treat the 
finite time horizon problem. The infinite time horizon problem is considered in the 
third section. Our numerical results are presented and discussed in Section 4, while 
a brief fifth section contains a summary and some concluding remarks. 

2. LQR Problems with Finite Time Horizon. In this section we consider 
the linear quadratic regulator (LQR) problem over a finite time interval. The basic 
notation and our general assumptions are introduced in the statements of both the 
continuous time and corresponding sampled time problems given below. The existence 
and uniqueness of the optimal control as well as its closed loop feedback structure can 
be obtained using a variety of approaches. Here we opt to consider the optimal 
control problem as the minimization of a strictly coercive quadratic form on the 
admissible control space. This approach yields an explicit representation for the 
solution of the usual Riccati equations (for both the continuous and sampled time 
problems) in terms of the underlying system and penalty operators which define the 
problems. Since the particular focus of our effort here is the consideration of sampled 
time problems as approximations to a continuous time problem, specialized notions 
and characterizations of convergence must be introduced. Once this is done, our 
fundamental result for the finite time horizon problem can be stated in terms of these 
specialized notions of convergence as follows. The convergence of the optimal control 
and the optimal feedback laws for the sampled systems to the optimal control and 
feedback law for the continuous time problem as the length of the sampling interval 
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tends to zero, follows directly from the convergence of the open-loop sampling of the 
underlying linear control system and quadratic performance index. We consider the 
open-loop sampling of the infinite dimensional LQR problem in an abstract setting so 
that our results can be applied to a wide range of sampling procedures. 

Let H and U be Hilbert spaces with inner products < *,• >h and < >u 
respectively. Let to,tf E R be given with to < tj, and let T = {T{t,s) : to < s < 
t < t/} be an evolution system on H. For each t E [*o>*/], let H(0 € L(U,H ), 
Q(t) E L(H), and R(t) E L(U ), and let G E L(H). We consider the continuous time 
LQR problem given by 

(P) Determine a control input u E L 2 {to,t f ;U) which minimizes the quadratic per- 
formance index 

J(u;to,x(t 0 ),G) = < Gx(t f ),x(t f ) > H 

+ f ' {< Q(*)x(t),x(t) >h + < R(t)u(t) , u(t) >u}dt 
Jt 0 

where for each t E the state x(t) E H is given by 

(2.1) x(t) = T(t,s)x(s) + J T(t, r)H(r)u(r)dr, to<s<t<tf. 

We make the following standard assumptions on the operator families {T, B, G, Q, R} 
which determine problem (P). 

(Cl) The evolution system T is strongly continuous on H and therefore is uniformly 
exponentially bounded, with constants M > 0 and uj E R. That is 

||rMIU(tf) < t 0 <s<t<tf. 

(C2) The operator valued functions B,Q, and R are strongly continuous and there- 
fore are uniformly bounded on [to> ^ /]• That is, there exists a constant C > 0 
for which 

\\B{t)\\i.(U,H) < Cy ||<2(0IU(*O — \W)\\l(u)<C, 

t E [toi t/]. 

(C3) The operator G and the operators Q(t) and R(t) for each t E [to,tf] are self- 
adjoint and nonnegative definite. Moreover, there exists a constant r > 0 for 
which R(t) > rl, t E [<o 5 </]- 

The strong continuity assumption in (C2) is not necessary for the well-posedness 
of the LQR problem. However, some assumptions on the continuity of the operators 
B,Q,R will be needed to obtain uniform convergence with respect to sampling. 

The closed-loop linear state feedback form of the solution to problem (P) can 
be shown to exist and be explicitly constructed by considering the minimization of 
appropriately constructed strictly coercive quadratic forms on the Hilbert spaces U s = 
L 2 (s,tf]U), s E (see, for example, [G]). Since it will play a prominent role in 
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our discussions to follow, we briefly outline this approach here. For each s G [to,tf\ 
define the operators 8, G L{H,U a ) and R a G L(U a ) by 

(2.2) = B(ty {T(t f ,t)'GT(t f ,s)+ T(r,,tyQ( V )T(r,,s)dr , } <*, 

for <f) G H, t G [s, t/J, and 

(2.3) (*.«,)(<) = R{t)u,(t) + B(t)’T{t t , iya y T{t,,t,)B{nKMdn 

+B(t)- y |r(n, t)‘QM [ T( V , r)B(r)«,(r)*j A,, 

for t G [s,t/] and u, G U s . It is not difficult to verify that the adjoint operator 
8* G L(U,,H) of B a is given by 

(2.4) 8 a u a = T(tf,s)*G J* f T(tf,t)B(t)u a (t)dt 

+ j, 1 r(r,a)*Q(r) T{T,rj)B(rt)u a (r})dri^ dr, 

and that for u a G U s , we have 

J(u,;s,x(s),G) =< GT(t/,s)x(s),r(t/,s)x(s) > H 

/ tf 

< Q(t)T(t,s)x(s),T(t,s)x(s) > H dt- < Rj 1 B a x(s),B a x(s) > u , 

+ < R a {u a + £7 l 8 4 x(s)),u 4 + Rj 1 B t x(s) > u . . 

It follows that for x(s) G H given, J(-; s, x(s), G) is minimized by choosing u, — u, = 
— R~ l Btx[a) G U a . We then obtain 

min J(-;5,x(s),G) = J(u,\ s, x(s), G) 

— < GT(t f ,s)x(s),T(tf,s)x(s) >h + f ' < Q(t)T(M)x(s),T(l,s)x(6) > H dt 

- < Rj 1 B a x(s),B a x(s) > u , 

= < n(s)x(s),x(s) > H 

where the self-adjoint operator valued function II : [to,t/] *— ► L(H) is defined by 

(2.5) n(s)<£ = T{t f , S yGT{t f ,s)<t> + J*' T{t,syQ(t)T{t,s)4>dt 

-BIR-^B,^ <j>e H. 

Using the definitions given above, the following theorem concerning the existence and 
characterization of the closed-loop solution to problem (P) can be established. 

THEOREM 2.1. Suppose that assumptions (Cl)-(C3) are satisfied. Then for 
any initial state x(t 0 ) G H given, there exists a unique solution u to problem (P). The 
optimal control u is given in linear state feedback form by 

u(t) = —J2(t)~ 1 B(t)*T[(t)x(t),t G [to,*/] 
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where x is the optimal trajectory. The operator valued function II is given by (2.5) 
and it is the unique self-adjoint solution to the Riccati integral equation 

(2.6) II(t) = T{t f ,tyGT{t f ,t) 

+ J*' T{T,t)* {g(r) - n(r)B(r)i2(r) _1 5(r)*II(r)}r(r,f)^, 

te[t Q ,t f ]. We have 

(2.7) min J{-]t 0 ,x{t 0 ),G) = J{u‘,t 0 ,x(t 0 ),G) =< n(t 0 )s(fo),z(*o) >h • 
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We consider next the discrete or sampled time problem. Let k 0 ,k f G Z with 
k f > k 0 and let h G R with h > 0. For k e Z with k 0 < k < k f - 1 let A h {k) G L(H), 
and let {T h (k,j) : k 0 < j < k < k,} be the discrete time evolution system on H given 

by 

jfc-1 

T h {k,k) = I, T h {k,j) = A h {k-l)-A h {k-2)---A h {j) = II ^*(0. k 0 <j<k<k f . 
( 2 . 8 ) 

Let {B h {k)} k k L~l {Q h {k)} k k L k l 0 , and {R h {k)} k ££ be sequences in L{U,H), L{H) and 
L{U) respectively, and let G h G L{H). The LQR problem is then given by 

(P A ) Determine a control input u h G li(ko,kf — 1; U) which minimizes the quadratic 
performance index 

Jh{uh', k 0 ,Xh{ko),Gh) =< GhXh{kf),Xh{kf) >h 

k/-l 

+ h ^2 {< Qh{k)x h {k),Xh(k) >h + < Rh{k)uh(k),u h (k) >rr} 

k=k o 

where for each k £ Z with k$ < k < fc/, the state (E H is given by 


*-i 

(2.9) x h (k) - T h {k,j)x h (j) + hY^T h (k,i + l)B h (i)u h (i), 

•=J 


for k 0 < j < k < kf. 

For the discrete time case, we make the following assumptions. 

(Dl) For each h > 0 the operators A h {k), B h [k), Qh{k), and Rh{k) are bounded in 
k for k 0 < k < k f - 1. Thus, there exists a constant C h for which 

- Ch ' \\ B h{k)\\L(U,H) < Ch, 

||<3h(^)||L(/r) < Ch, ||fl*(fc)IU(P) ^ Ch, 
for k 0 < k < kf — 1. 
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(D2) The operator G h and the operators £?*(*) and Rh(k) for k 0 < k < k f - 1 are 
self-adjoint and nonnegative. Moreover, there exists a constant > 0 for 
which R h (k) > r h I, k 0 < k < k f - 1. 

Note that assumption (Dl) together with (2.8) yield that the discrete time evolution 
system {Th(k,j) : k 0 < j < k < kf} is uniformly exponentially bounded with 

m(MIU(tf) < C k h ~\ k 0 < j < k < kf. 

Note also that the discrete - time evolution equation (2.9) is equivalent to the discrete 
time dynamical system given by 

(2.10) x h (k + 1) = A h (k)x h (k) + hB h {k)u h {k), k 0 < k < kf - l,x h (k 0 ) e H. 

For each h > 0 and j = * 0 , k 0 + 1, • • • , k f - 1, let U k j = h{j, k f - 1; U) endowed 
with the inner product 


k /~ 1 

< u h,j,Vh,j >u K f= h.J2< u hJ (k),v hij ( k) > v . 

k=j 


Define the operators B h j <E L(H,U h j) and Z h j € L(U hJ ) by 


(2.11) ( Bk,j<f>)[k ) — Bh(k)*Th(kf, k + l)*GhT/ l (kf,j)(f> 

[ */- 1 

+Bk{k) j h 52 T h {i, k + l)*Q h (i)Tk(i,j) l 


for 4> € H, k = j,j + 1, . . . ,k f - 1, and 
(2.12) (£*,,■«*,•)(*) =^(A:K, y (fc) 

+B h {k)'T h {kf,k + iyG h h 52 T h (kf,i+l)B h {i)u hJ (i) 


kf - 1 


*=; 


t f ~i 


i— 1 


+B h (k)'h £ T h {i,k + l)'Q h (i) \h^T h {iJ + l)B h {l)u h j{l) , 


*=fc+l 


1=3 


u h,j € Ukj, k - j,j + 1,. • • ,k f - 1, respectively, where in the above expressions and 
throughout the remainder of the paper we adopt the convention that J2i= a. = 0 
whenever u < fi. It is not difficult to verify that the adjoint of Bhj, the operator 
Kj £ L(U h j,H ) is given by 


kf~ 1 


(2.i3) Kj u hj - Tk{kf,j)*G h h J2 Tk(k f ,k+l)B h (k)u h j(k) 

k=j 

kf ~ l ( k ~ 1 
^ XI T k (k,j) Qh{k) < h 52 Th(k, i + 1 ) Bk ( t) tt h y ( t ) 1 . 


k=j + 1 


* =3 
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for u hJ G U h j- 

Proceeding as we did in the continuous time case, we find that for j = ko, ■ • • , kf — 
1, Xh(j) € H, and u h>j G U’ hij 

Jh(uhj;j,x h (j),G h ) = < G h T h (kf,j)x h (j),T h (k f ,j)x h {j) > H 

k f -l 

+^E < Qh{k)T h (k,j)x h (j),T h (k,j)x h (j) > H 

k=j 

~ < &kjBKjx k (j),BkjZ k {j) >u kJ 

+ < Zh,j{u h ,j + Z^B hJ x h {j)),u htj + JZj£8 h jX k (j) > Ukj , 

where the existence of the inverse of Z k j is quaranteed by assumption (D2). It 
is immediately clear that for j G Z with j G [ko,kf — l] and x k (j) G H given, 
Jh{‘\j,Xh{j), Gh.) is minimized when u k j = = — k k j&hjX k {j). It follows that 

Jj %h{j) )> G k } J k [u k j, j, x/j(j) , G/f) 

“ fc .y 

= < G H T h (kf,j)x h (j),T h (k f ,j)x h (j) > H 

kf-l 

+h X) < Qh{k)T h (k,j)x h (j),T h (k,j)x h {j) > H 

k=j 

- < ZkjBkj*k{j),B kJ x h (j) > UkJ 
= <n k {j)x k (j),x k (j) > H , 

where the sequence of self-adjoint operators in L(H), {n h (fc)} t { :Jfco , are given by 

*/- 1 

(2.14) n h (j)<f> = T k (kfJYG k T k (kf,j)4 + h X) T h {kjyQ k (k)T h {k,j)<j> 

k=j 

for k = ko, • • • , kf — 1 and <f> G H . We note that it is completely consistent to define 

n k (*/) = g h . 

Using the above definitions, it is possible to establish the following well known 
result (see, for example, [LCB], [Z], and [GR]) for the discrete time LQR problem 

(Pa). 

THEOREM 2.2. Suppose that assumptions (Dl) and (D2) are satisfied. Then for 
any given initial state Xh(ko) G H there exists a unique solution U/, G h{ko, kf — 1; U) 
to problem (Ph). It is given in linear state feedback form by 

Uh(k) - -R^h)' 1 B h (k)*n h (k + l)^4 h (fc)x fc (A:), k = k 0 , ■ • • , k f - 1 , 

where Rh(k) = Rh(k) hBh(k)'Tlh(k + l)Bh(k), for k = ko, • • • , kf — 1, and the optimal 
trajectory x k is given by (2.9) ( equivalently (2.10)) with u k — u k . The sequence of 
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operators in L(H), {II/, (A:) are given by (2.14) with IIfc(fc/) = Gh and can be 
obtained recursively via the Riccati difference equation 

(2.15) I l h (k) = Ah(kyjlh(k + l)A/,(/c) 

-hA h {kyn h {k + ^B^RHiky^Hikyu^k + i)A h {k) 

+hQh(k), 

k = kf — 1, ■ • • , ho, n^(/c /) = Gh- We have 

(2.16) min J h (-, k 0 ,x h (k 0 ),G h ) = J h {u h ; k 0 , x h (k 0 ), G fc ) 

U h,k 0 

= < Uh{k 0 )x h (k Q ),x h (k 0 ) > H . 


For appropriate choices of the families of operators Th, Bh,Qh, and Rh, we are 
interested in studying the convergence of solutions to the problems (P/,) to the solution 
of problem (P) as the length of the sampling interval, h, tends toward zero. In 
particular, we want to investigate the convergence of the discrete families of Riccati 
operators {]!/,(&) : k 0 < k < kf } to the continuous family of operators {II(t) : to < 
t < t f }. 

In ~ 'der to reduce the necessary degree of technical detail, we make the simplifying 
assumption that t 0 = 0. There is of course no loss of generality in doing this since 
any system can be transformed to one on a time interval starting at the origin. Set 
A: 0 = 0 and for each h > 0 let kf = kf h — [ tf/h ] where for a 6 R, [a] is used to 
denote the greatest integer less than or equal to a. Let tj t h — hkf t h and note that 
limfc_ 0 + tf,h = tf- 

In order to compare discrete and continuous families of operators, it is useful to 
identify certain / 2 sequence spaces with subspaces of L 2 . For X a Hilbert space and 
all h > 0, let L 2 > h{Q,tf j l ] X) be the subspace of L 2 (0, tf t h\X) defined by 
L 2 /,(0, tfy, X) = {<f> € L 2 (0, tf t h", X) : <f> is constant on each of the intervals 
[0, h), [h, 2h), • • • , [(fc M - l)/t, «/,*)}. 

Note that the subspace -L 2 ^(0,f/ ^; X) of L 2 (0, £//,; X) is isometrically isomorphic to 
the space / 2 (0 ,kf t h — 1;X) endowed with the inner product 

< {«&■*, >= h e <*i,*s>x. 

;=o 

Let U = L 2 (0, £/;{/) and let Uh = L 2i /,(0, t/y, U). Let P /, 6 L(U,Uh) be the 
orthogonal projection-like mapping of U onto Uh defined by 

(P h <t>){t) = £ {<f> h ) jX i,{t), 0 <t<t ft h, 

;=o 

for <f> € U where for j = 0, 1, • • • , kf t h — 1, Xi is the characteristic function for the 
interval 7, = [ jh , (j + l)/i) and 

= h~ 1 f <t>{t)dt. 

Jj i 
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It is not difficult to show (see Appendix A) that 

(i) the net {||Pfc||L(a 1 u fc )}/»>o is uniformly bounded; 

(ii) limfc_o+ ||iW|k = \\<f>\\u, <t> £ U, and 

(iii) for each ip £ Uh there exists a <f> € U such that ip = P h <p and ||d>||a = ||^»||u fc . 

Following Kato [K, §IX.4] we say that a net {<ph}h>o^<Ph € Uh converges to <f> £ U 
(■ <t>h -*• <P, or lim/ l _o+ <p h = <p ) if 

lim || <f> h - P h <t>\\u h = 0. 

A— ♦■O' 

Also, if for h > 0, £ L(Uh), then we say that converges strongly to $ £ L(U) if 

$hPh<P —* $<f>, <p€U\ that is if 

lim || $hPh<P ~ Ph$4>\\u k = o, <PeU. 

h — ►O' 

With strong operator convergence defined in this way, it can be shown that $ k P k (p —+ 
$<f>, <p £ U implies that the net {||$h||L(a k )} is uniformly bounded and that if $hPh<P 
$<£,and ^hPhP — » ► ^<P, <f> € U, then $hVhPh<P <f> € U, etc. We note of course 

that an analogous definition of strong convergence can be made for bounded operators 
having only one or the other of its domain and co-domain being Uh- That is, for 
example, if £ L(X,Uh) and $ £ L(X, U) where X is a normed linear space, then 
we say that converges strongly to $ if $/,x — + $x, x £ X, or 

lim ||$fcX - P>,$xL. = 0. 
h— 0 + 

Following the treatment of discrete semigroups in Kato [K], we make the following 
formal definition. 

DEFINITION 2.1. The discrete time families of bounded linear operators = 
{*h{k n ,k n . l ,-'-,ki) : 0 < k x < k 2 < ••• < k n < K h }, h 0 from a Banach space 
X into a Banach space Y will be said to (strongly) approximate a continuous time 
family of operators $ = {$(i n , t n _j, • • • , ti) : 0 < t x < t 2 < • • • < t n < T} with 
$(*«»• ’’.ti) € L(X,Y) for t = {t n ,-“,t i) £ A (n,T) = {(t n ,t n - 1} • • • ,*i) £ R n : 0 < 
< h < • ■ • < t n < T}, at t = (f n , ••• ,ti) £ A(n,T), if 

(i) There exists at least one net of multi-indices {kh}h> o> k = ( k n h , • • • , k\,h) £ Z n 
with 0 < k i h < • • • < k n h < Kh and limfc_ 0 + hkh = t. 

(ii) For all nets {kh}h>o, satisfying (i) above, 

lim ||$fc(£fc)x — $(f)x||y = 0,x £ X. 

h— 0 + 

The families h > 0 will be said to approximate $ on the set A(n, T), if Kh = [T/h] 
and if approximates $ at each t £ A(n,T). 

When the discrete time families h > 0 approximate the continuous time family 
$ at time t (on the set A (n,T)) we shall write at time t (on the set A(n,T)). 

Definition 2.2. For h > o and = {**(*»,*„-!, ■■•,*!) : 0 < k X < k 2 < 
••• < k n < Kh} a discrete time family of bounded linear operators, we define an 
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associated continuous time family of operators, • * • , £ 1 ) : 0 < t x < 

t 2 + h < ••• < t n + h < (K h + l)h} via $ h (t n , • • • ,ti) = $ k ([t n /h},- ■ ■ ,[t i//t]) for 
t = (t n , * * * , i 1 ) E A^(n, X/,) = {(£„, £ n _ i, • • • , £ 1 ) E R n : 0 < fi < t 2 + h < • ■ • < 
t n + h < (Kh + l)h}. 

Note that when Kh = [T/h], A (n,T) C A/,(n, if*) for all h > 0. 

The proof of the following theorem can be argued in much the same manner as 
were the proofs of Lemmas EX. 3. 4 and IX. 3.5 in Kato [K]. 

THEOREM 2.3. Suppose that the continuous time family of bounded linear opera- 
tors $ is strongly continuous on A(n,T) and that h > 0 are discrete time families 
for which $ on the set A (n,T). Suppose further that for each h > 0, is 

the continuous time family on A h(n,Kh) corresponding to the discrete time family 
constructed according to Definition 2.2 above. Then 

(i) The families h > 0 are uniformly bounded in h in L(X,Y); that is there 
exists a constant M > 0 independent of h for which 

< M, 0 < fci < k 2 < • ■ ■ < k n < Kh, h > 0, 
(it) $h — * $ uniformly, in t for t € A (n,T); that is 

lim ||$/,(£):e — $(£)a;||y = 0, x € X, 

h— o+ 

uniformly in t for t — ( t n , • ■ • , £i) E A(n, T). 

Conversely, if Kh = [T /h] and $h — * $ uniformly in t for t 6 A (n,T), then — 1 ► $ 

on the set A (n, T). 

Let the continuous time families T = { T(t,s ) : 0 < s < t < T} C L(H), 
B = {£(£) : 0 <£<£/} C L(U,H), Q = {Q(£) : 0 <£<£/} C L(H) and 
R = {R(t) : 0 <£<£/} C L(U ) be as given in the statement of the continuous time 
LQR problem (P) (i.e., in particular assume that the conditions (Cl)-(C3) hold). 
For h > 0, let kfh = [tf/h ) and let Ah = {j4^(A:) : 0 < k < k/^ — 1} C L(H ), 
B h = [B h (k) : 0 <k < k fth - 1} C L(U,H), Q h = {Qh(A:) : 0 < k < k f<h - 1} C L(H), 
and R h = {Rh(k) : 0 < k < kf h — 1} C L(U) be discrete time families of bounded 
linear operators which satisfy conditions (Dl) and (D2) and which satisfy the following 
conditions. 

(Al) B h — > B, Qh — » Q, Rh — ► R, and B £ — ► B* on the set A(l,i/) where B* = 

{B(0* : 0 <£<£/} and B* h = {B h (ky :0<k< k fh }. 

(A2) (a) (Stability) The discrete time families of operators Th = { Th(k,j ) : 0<j < 
k < k fi h} C L(H) given by 

f *-i 

n^o. i<K 

I, j — k 

are uniformly bounded in L(H) for h > 0. 
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(b) (Consistency) 

lim hf h {t + h,t)<l>-T{t + h,t)<t>\\=0, 4>e H, 

h-* o+ h 


and 

lim 7 \\T k {t + h, t)*4> - T(t + h, t)*<f>\\ = 0, <j> 6 H, 

h—> o+ h 

uniformly in t for t E (0,t/j. 

(A3) The scalars r h given in the statement of condition (D2) are bounded away from 
zero uniformly in h. That is r/, > r > 0, h > 0. 

LEMMA 2.1. Condition (A2) implies that T h -* T and TjH -> T * on the set 
A(2 ,t f ). 

Proof. We consider the convergence Th —* T only; the adjoint convergence is com- 
pletely analogous. Following the proof of the well known Lax-Equivalence Theorem 
[RM], the result is an immediate consequence of condition (A2), the strong continuity 
of the continuous time family T , and the identity 


y Th(k, t + 1){A*(0 - T((i + l)h,ih)}T(ih t m, k> 
T h {k,j)<t>-T{kh,jh)4>= fr' 

{ 0 , k = 

0 < j < k < kf h, d> € H . r ? 

We shall also assume that G e L(H) is as in condition (C3) and that for each 

h > 0 the operator G h € L{H ) satisfies condition (D2). We require that the additional 
approximation condition 
(A4) lim^_o+ Ghd> = G<j > , <p £ H, 
be satisfied as well. 

For h > 0 and s € [0, £ /] define Bh, a € L(H,Uh) by 


(2.17) 


(B h »(0 = 


For £ E [0,£/,>i — h] : 

X|[ W t + hya k T K (t,,H , «) 

+ / M + h)’Qh(ri)f h (ri,s)dri}4>, 

J\(t+h)/h]h 


For t E - h, £/,/»] : 
k (B^ t3 ^) (t/,A — h), 


when s E [0,t /lh ), and by (S M ) = 0 when s E [*/.*,*/], for <t> E H . Note that for 
j = 0, 1, 2, • • • , ic fih — 1, k = j, j + 1, ■ • • , k fih - 1, and <f> E H 


(2.18) ( B h>a <t>){t ) = ( 8h,j<l>)(k ), 

for s E [jh,(j + l)/i) and t E [kh, (k + l)h), where for j = 0, 1, 2, • • • , k f>h - 1, 
B h j e L{U,U h j) is given by (2.11). 
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Similarly, for 3 G [O,*,] define B' h>a G L(U h ,H) by 


(2.19) B* h<a u h = f h (t Lh ,syG h [ tf ' k f h {t fih ,t + k)B h (t)u h (t)dt 

J[s/h}h 

r*f* ~ ( r[r/h]h „ ) 

+ /(«.»)/»! J h ^ T ’ 3) ‘ QhM \i mh T ^,1 + h)B h Mu h (n)d^ dr, 

when 5 G [0,f /iA ), and by B^ a u h = 0 when s 6 [*/,*,«/] for u h G Note that for 
J = 1? 2, • • • , kf t k — 1 and Uhj G Uhj we have 

( 2 - 2 °) 8m«* = 8»> w 

for s G [jh, (j + l)/i) when u A G is given by 


( 2 . 21 ) 



0 < t < jh 

kh < t < (A; + l)/i, 


^ an< ^ &hj € L(Uhj,H) is given by (2.13). Note also that 

&h,$ (^ m ) • That is that B^ a G L(Uh->H') given by (2.19) is the Hilbert space 

adjoint of the operator B h>a G L{H,U h ) given by (2.17) for all s G [(),«,]. 

For 3 G [0, t f ) define k K ,s € L[U h ) by 


(2.22) (**,.**)(*) 




For t G [0, ^/,/t — h\ : 

Rh{t)u h (t) + X[[ a /A]M/,fc]( f )i?>,(0* 


Th{tf,h,t + h)*G h f T h (t fih ,r) + h)B h (r})u h {r))dr) 

J\8lh\h 
M 


\fc»» T ^ t+k r i >'W 

r\n/h]h . 

/. /ui. T + h ) B h{T)u h {T)dT 

J[8fh\h 


drf 


)■ 


For t G [*/,* - h,t fih ] : 
„ (£*,««*)(*/,* - h), 


when s G (0,t M ) and by (£ M u h )(t) = &(*)«*(*), 0 < t < t fik , when 3 G for 

u A G U h . We again have that for j = 0, 1, 2, • • • , k fih - 1, k = j, j + 1, • • • , k f>h - 1, and 

u h j e U h j, 


(2.23) 


{Rk,sUh){t) — (khj u hj){k) 


for s G (jh, (j + l)/i) and t G [kh, (k + l)/i), where G Uh is given by (2.21) and 
€ L(U h j) is given by (2.12). We note also that k K , is self-adjoint and positive 
definite on Uh and that if we let Uhj denote the subspace of U k obtained from Uhj via 
the natural embedding (i.e. via (2.21)), then, R h ,3 is a bijection from Uhj onto Uhj- 
It follows therefore from (2.17), (2.19), (2.22) that for j = 0, 1, 2, • • • , k fih - 1 


(2.24) 


&hj%hj&hj<t > ~ B ha k h ^ a Bh,3j> 
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for each 4> E H and all s E [jh, ( j + l)/i), and that 
(2.25) KKM = 0 


for all 6 6 [t fj h,t f }. 

Setting II fc (s) = Jl h {k), kh < s < {k + 1 )h, for 5 E from (2.11), (2.12), 

(2.13), (2.14), and (2.24) we find that 

(2.26) ri„( 5 )0 = f h {t fih ,syG h f h {t fth ,s)<t> 

+ f ! ' k f h {t,sYQ h {t)f h {t,s)4>dt - 

J[sjh\h 

for each <f> E H. Note that (2.25) implies that rU(t) = G for t E [*/,/»>*/]• 

For B, E L(H, U a ), Z a E L{U a ),_ and Bf E L(U„H) given by (2.2), (2.3), and (2.4), 
respectively, define B a E L(H,U), Z a E L(U), and B* E L(U,H) by 


(2.27) 

(M(0 = | 

\ 0, 

l (*.*)(*). 

0 <t < s, 
s<t<t f , 

(2.28) 

Cm 

II 

\ R{t)u{t), 
{ (*.«)(*). 

0 < t < s, 
s <t <tf, 

and 




(2.29) 


Bfu = B*u 



for 4> E H and uE.U. It is not difficult to show that B* — (B a )* (i.e. that S* E L(U,H) 
is the Hilbert space adjoint of B a E L(H, U)), that Z a is self-adjoint positive definite on 
U, and that if we let U a denote the subspace of U obtained via the natural embedding 
of U a into l/, then R, is a bijection from U, onto U a . Consequently, if follows that 

(2.30) B;R; 1 B a ct> = B;z; 1 B a <i> 
for all <j> E H and s E [0,t/]. From (2.5) we obtain that 

(2.31) n [s)4> = T{t f ,sYGT{t f ,s)4> + j tf T{t,sYQ(t)T{t,s)<f>dt 

-B;k~ s l B a <j>, 

for all <f> E H and s E [0, i /] . 

Our fundamental convergence or approximation result for the finite time horizon 
problem is given in the following theorem and its corollary. 

THEOREM 2.4. Suppose that the families of operators {T,B,Q,R} satisfy con- 
ditions (Cl)-(C3) and that for all h > 0, the families of operators {Th, Bh,Qh, Rh} 
satisfy conditions (Dl) and (D2). Suppose further that the approximation assumptions 
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(A1)-(A4) are satisfied. Then the discrete time family of operators II* = (II*(fc) : 0 < 
k < kf'h} given by (2.14) or (2.15) strongly approximates the continuous time family 
of operators II = {II (i ) : 0 < t < tf } given by (2.5) or (2.6) on the set A(1 ,tf). That 
is, n*->n on the set A(l,i/). 

Proof. The desired result will follow from Theorem 2.3 if we can argue that 
lim*_ 0 + ft*(t)<£ = II(i)<£, uniformly in t, for t G (0, t 7 ], for each <j>€ H, where ft* and 
II are given by (2.26) and (2.31), respectively. 

From assumption (A3), we have that the operators £** are bounded uniformly 
in h > 0 and s G [0, tf\. Somewhat technical, but rather elementary arguments can 
be used to show that Bh, s 4> B,4>, for all <j> G H, ^* i 3 P*u —* R,u for u G U, and 
B^ s P h u — ► B*u, for u € U, uniformly in s for s G [0,t/], where B*, a , B,, Zh,,, R,, B*, a 
and S; are given by (2.17), (2.27), (2.22), (2.28), (2.19), and (2.29) respectively (see 
Appendix B). This together with the identity 

zt^Ph - PhZ; 1 = z-^{Zh,sPh - PhR,}z ; 1 

yield that lim*^ 0 + = B*,k~ l B,4 > , for <f>G H, uniformly in s for s E [0,i/]. 

The desired convergence can then be obtained from assumptions (Al), (A2), (A4) 
and equations (2.26), (2.31). O 

Let F = {F(t) : 0 <t <tf} and 5 = (5(t,s) : 0 < s < t < t f } be respectively the 
continuous time families of optimal closed-loop feedback gain operators and optimal 
closed-loop state transition operators for the continuous time LQR problem (P). That 
is, for t G [0, t f] 

F(t) = R{t)- 1 B{tyn{t)GL(H,U), 

and for 0 < s < t < tf 

(2.32) S(t,s)4> = T{t,s)<j>- j T(t,r))B(r])F{ri)S{ri,s)<f>dr) 

= T(t, s)<f> - J* T{t , r])B(r))(R~ 1 B*4>)(r])dr], 

for <j> G H (see [G]). Similarly, for the discrete time problem, let the discrete time 
families, F* = (P*(/c) : 0 < k < kf * — 1} C L(H, U) and 5* = {S^(k, j) : 0 < j < k < 
kf'h.} C L(H) be given by 

F h (k) = Rhtk)' 1 B h {kyil h (k + l)A h {k), 

where 

R h (k) = R h {k) + hB h {kyU h {k + 1 )B h {k), 
k = 0, 1, • • • , k f h - 1, and 

Jfc-1 

(2.33) S h {k,j)4> = T h {k,j)4> - hJ2 T h {k,i -I- l)B h [i)F h (i)S h {i,j)<f> 

i-j 

= r*(fc,i)^-/i£V*(fc,t>i)B*(t)(^;.B;^)(0, 

i=j 
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o < j < k < for (f) e H. 

COROLLARY 2.1. Suppose that the hypotheses of Theorem 2-4 above are satis- 
fied and let {«, x} and {u h ,x h } be the optimal control/trajectory pairs for the LQR 
problems (P) and (P fc ), respectively, corresponding to the initial data x(0) = x h {0) = 
xo E H . Then 

(i) F h - F; 

(ii) S h - S; 

(Hi) lim/,_o+ — = 0, and lim/,_,o+ || x ft(^«) — x (0llw — / or ^ ^ 

and for all nets {kh}h > o for which limh—o+ hkh = t. 

(iv) lim h _ 0 + J h = J. 

Proof. Statement (i) and (iv) (recall (2.7) and (2.16)) are immediate consequences 
of Theorem 2.4. Statement (iii) follows from statement (i) and (ii) since u(t) = 
— F(t)x(t), x{t) = S(t,0)x 0 , t e and u h {k) = -F h {k)x h {k), 0 < k < k f , k - 1, 

x4fc) = Sh{k, Q)x 0 , 0 < k < k fth . Thus we need only to verify statement (ii). 

We rewrite (2.32) as 

S{t,s)<f> = T{t,s)4> - J* T{t,v)B{v){Z; 1 8;<l>){ri)dr,, 
and from (2.33) we obtain 

Sh(t, s)<f> = Th(t, s)<p — f Th(t, r) + h)Bh{r}){R h ),B h a <i>]{rj)dri. 

J[s/k]h 

The result now follows as in the proof of Theorem 2.4 □ 

REMARK In actual practice, given the continuous time LQR problem (P), the 
net of discrete time problems {(P^)} is typically obtained by considering zero-order 
hold (i.e. piecewise constant) control inputs and output sampling. In this case we 
would obtain A h (k) = T((k + l)/i, kh), B^(k) = h 1 ^ T((k + l)h, s)B(s)ds, 

Qh{k) = h- l f^ l)h Q{s)ds> R h {k) = h~ l R{s)ds, and G h = G. When con- 
ditions (C1)-(C3) on the continuous time families T, and R are satisfied, it is 

immediately clear that the discrete time families B^^Qh an( I Rhi and the oper- 
ator G h satisfy conditions (Dl) and (D2) and the approximation conditions (Al)- 
(A4). More generally, other discretizations are also admissible. For example, in the 
time invariant case, the semigroup {T(i) * t > 0} could be discretely approximated 
using A-stable Pade approximants to the exponential (see [HK]). In particular, if 
T(t) = exp(tA), t > 0, then one might set T h {k) = (J - hA)~ k (implicit Euler) or 
T h (k) = (I - hA/2)~ k [I + hA/2) k (Crank-Nicolson). The stability and consistency 
of these discretizations (i.e. assumption (A2)) can be verified using the theory and 
techniques developed in [HK]. Finally, along these same lines, the convergence of si- 
multaneous but independent state and time discretization in the context of the LQR 
theory should also be looked at. We note that appropriately “coupled state and 
time discretization can be handled using the theory and framework which has been 
developed above. 
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3. The Infinite Time Horizon Problem. The linear quadratic regulator prob- 
lem over an infinite time interval can be viewed as an extension of the finite time 
interval problem. The state equations (2.1) and (2.9) governing the dynamics of the 
continuous time and discrete time control systems, respectively, remain the same. The 
continuous and discrete time operator families {T,B,Q,R}, and {T h , B h , Qh, Rh} are 
assumed to be defined on the infinite time intervals (to,+oo) C R and [kc, +oo) C Z, 
respectively. The cost functionals are taken to be 

r oo 

(3.1) ./ooK*o,:e(*o)) = / {< Q{t)x{t),x{t) > H + < R(t)u(t),u(t) > v }dt 

= lim J(u;to,x(t 0 ),0) 

t f — ►OO 

and 

OO 

Jh,co{uh\ k 0 ,x h {k 0 )) = h {< Qh(k)x h (k),x h (k) > H + < R h {k)u h (k), u h (k) >*/} 

k=ko 

(3.2) = lim J h (u h ;k 0 ,x h (k 0 ),Q) 

kf—*oo 

Under appropriate stabilizability and detectability assumptions on the continu- 
ous time and the discrete time control systems, the existence and the uniqueness of 
the optimal controls u,Uh minimizing (3.1) and (3.2), respectively, can be obtained. 
Moreover, these optimal controls can be written in a closed-loop state feedback form 
(see Theorem 3.1 below). We are again interested in investigating the convergence of 
the optimal controls and the optimal feedback laws for the sampled systems as the 
length of the sampling interval tends toward zero. 

Our fundamental result can be outlined as follows. Assume that the conditions 
(Al)-(A4) for the convergence of the open-loop control problems are satisfied on 
every finite time interval [i 0 , */]• Suppose further that the stabilizability and the 
detectability of the continuous time system are uniformly preserved by the sampled 
time systems (see Definition 3.3-(iii) and 3.4-(iii)). Then the optimal controls u 
and the optimal state feedback laws Fh for the sampled time systems converge to 
the optimal control u and optimal feedback law F for the continuous time system, 
respectively, as the length, h, of the sampling interval tends toward zero. We note 
that the problem of uniform preservation of stabilizability and detectability under 
sampling is in general, a difficult one. Here we shall treat this question only in a 
limited sense. We shall have to assume finite rank feedback stabilizability and finite 
rank detectability (see Condition (F)) for the continuous time system, although we 
have some conjectures about other reasonably broad classes of systems for which these 
conditions can be verified. We address this question in greater detail below. 

As in the finite time horizon problems, the functionals J ^ and J h oo can be viewed 
as quadratic forms on L 2 (t 0 , oo; Z7) and / 2 (fc 0 , oo; U), respectively. However, since J 
and Jh,oo are, in general, not bounded (for example, the uncontrolled system may 
not be asymptotically stable, hence the cost for the control input u = 0 may be 
infinity), one must deal with some rather tedious technical details. Therefore, the 
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infinite horizon LQR problems — 

finite time horizon problems with the fina im , then be obtained 

and the uniqueness of the optimal controls optimal feedback 

by considering the limiting behavior of the opt f th funct ionals J*, 

laws for the finite time horizon as the limit of the 

and J K oo, ^ is natural to view em ^ operators G and G h taken to be 

finite horizon problems with stories x(t 1 and x h (k ) are known to 

zero - However, we note that if « e opt.^ tra^ t M ^ X W then 

be asymptotically stable (i.e., x( ) , ( ’ , .1 y m ^ 0 f fi n it e horizon 

the infinite horizon LQR problem can also . be " u ° me henc eforth, 

problems with G*,G > 0. Once again, for simplicity, 

without lost of generality, that t 0 - k 0 - 0. 

DEFINITION 3.1. (Cost functional stabilizability) m is said to 

(il The continuous time system associated with the p P ’ 

( ’ be cost functional stabilizable with respect to the port ormance 

a >‘o; there exists a control input £ L.C, £0 with f f W 

Hi) The sampled time svstem associated with the operator pan {Th, Bh) said 
(U) be s ”u ctiona, stabilizable with respect to the discrete perfo™mr « ,nd x 
7 given by (3.2), if for each * 6 H, there ex.sts a constant M„ f) such 
that for any j > 0, there exists a control input sequence u h ., 6 3 (j,oo, 

with Jh } oo { u h,j S J ? ^0 — fiinrtional stabilizable for 

(m) T ”’ < rrs,*^ e “-slants MhW defined in (ii) are 

For any given final time ty and ** ““ ft 
denote the Riccati opera^om # ^ Q = Gh = 0, using (2.7) and 

penalty operators G and G h , respe y t > 0 and fc > 0, 

to ifil it ic pasv to verify that see for example, DljJ tor eacn giv _ 

(2.16), it is easy to verny * V „ (fc;0 ) are nondecreasing, self-adjoint, 

the functions *, n t/ (f,0) and m cost functi onal stabilizability of 

nonnegative operator valued func • nrovides an upper bound for 

the continuous and discrete time control systems then provides upp 

n t/ and n fc ,* /ik . Indeed, we have 

< flr.lt; 0)^,<^ >h< M[4), 


and 


< 


n fciJtM (fc;0)<M >H< Mh{4)> 
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operator valued functions bv IT anH T7 ( 

• r y oo( i ) nd Il/j oof*, OJ, respectively. The existenrp pnr? 

ztXn‘rn, ,ons to th ,: r inuous and “ «“ « cc izizt 

[LCsi! ( HH," Ind [Z [ 8 kn ° Wn f ° r ™» le . ! BW 1- [OJ, [GR], 

THEOREM 3.1. Assume that the continuous time system and the samnled time 
*and7> cJ a J 1 ' SU ^ Ctently Smail are cost functional stabilizable. Then for any s > 0 
u and- ’ ^h - T m StQteS ^ = * ° nd Xj ’ h = tkere exist uniyue °P tima l controls 
ZvZ Tf th ‘ tT /Uncti ° nals over Li(s,oo;U) and 

respectm,y - The " " « *• -*•» 


and 


6 W = = ~F(t)t(t), 

Mk) = -R.(k)-'B h (kyTl h Mk + = -F h (k)z,(k), 

hB 7lrVn nd l f* TJi e ^pooding optimal trajectories and &(*) = RJ k) + 

2 &E z:::zr^z » 

(3-3) n TO (s; 0)0 = T{t, 5 )*n oo (i; 0 )T(f, s )0 

+ L T ( T ' s y^ T ) -^{r-,Q){BR-'B*){r) n oo (r;O)]2’(r, 5 )0rfr, 

Is bounded f 6 A(2,oo). Similarly, the operator-valued sequence n fcoo (-0) 

W b ° Unded f° r0 <*<o o and satisfies the Riccati difference equation ( ’ J 

(3- 4 ) n* iOO (A; 0 ) =i4*(Ar)*n* i0O (fc + l; 0 )A fc (*)+Ag A (A) 

-M*(*)*n^(* + l;0)B*(*)A(*)-^*(*)*n fci0O (ib + l;0)il*(ife)^ 

Ven \ S h ampUd \ ttme S , ySt * ms are ^iformly cost functional stabilizable for 0 < h < h 0 

X* sr °:i a <t~ s n ‘“ (;o) are amf ° rm,y 

We assume that the general conditions UiWa^w *u 
open .cop , proh.ems hoId 8 any siv *JZ7e 

to n lt-n) • tk ’ n 0 vlous tuffi c ient c °ndition for the convergence of fL (t- 01 

n " ( ’° ‘ he come ^nce of n,,(t iG ) to n„((;0) and the uniform convergence 

18 



in h of n M „(t;G k ) to fu,»(t;0) (with = \t,lh]) an t, tends to infinity for some 
G > 0 and corresponding G h > 0. Indeed, from the triangle inequality, for <j> H , we 

have 

+ ||fi M/k (t; G h )* - n t/ (t; G)<f>\ \ H + ||n t/ (t; G)<f> - IT*, (t; O)0|| H . 


Then for an arbitrary e > 0, a sufficiently large t f can be chosen such that the first 
and the last terms on the right hand side of the above inequality are smaller than e/ 3 
for all h. By applying the theory of the previous section on the interval [0,t/j, there 
exists h 0 > 0 small enough* such that for all 0 < h < ho, the second term on the rig t 
hand side of the above inequality is bounded by e/3. Thus, the desired convergence 

immediately follows. . . 

As we have pointed out, if the trajectories of the systems are asymptotically stable, 

then as t f tends to infinity, the cost functionals Joo and are also limits of the cost 
functionals J, J h for the finite time interval problems on [0, t f ] with final state penalties 
G and G h different from zero. In particular, if the optimal trajectory of the infinite 
horizon problem is asymptotically stable, the convergence rates of J h {u h ; k, 4>, G k ) =< 
n h<kfh {k]G h )<j>,<j> > H with G h > M h {4>) and J(u;f,&G) =< n t{ {t\G)<f>,<t> >„ with 
G > M{4>) can be estimated by the decay rate of the optimal trajectory x for the 
infinite horizon problem. Toward this end, let S = {S{t,s) : 0 < s < t < oo} be the 
continuous time evolution system given by 

(3.5) S(t,s)4> = T(t,s)<f> — f* T(t, t) B (t) F (t) S (t , s)<j>ds, for (f> E H. 

The evolution system S is also referred to as the perturbation of T by -BF. It 
is not difficult to verify that S(t,0)<j> corresponds to the optimal trajectory for the 
continuous time infinite horizon problem with initial state <f> e H. Similarly, let the 
discrete time evolution system S h = {S h (i,j) : 0 < j < t < oo} be defined as 

| 7 \~ h 

(3.6 £/,(*, j) = | f]‘ fc -i{A fc (fc) - hBhityRkiky'Bhikyilh.ooik + 1;0 )A h (/t)}, i > J • 

Thus, S h (k, 0)4> is the optimal trajectory for the discrete time infinite horizon problem 
with initial state <f) G H . 

DEFINITION 3.2. (Exponential stability of the optimal feedback systems) 

(i) The optimal continuous time feedback system (3.5) is said to be exponentially 
stable, if there exist constants M and a > 0 such that for all 0 < s < t < oo, 
\\S{t,s)\\ L[H) <Mexv{-a{t-s)}. 

(ii) The discrete time optimal feedback system (3.6) is said to be exponentially 
stable, if there exist constants M h and a h > 0 such that, for all 0 < j < i < oo, 
||Sfc(*,y)||i(ff) < M h exp{-a h {i - j)h}. 
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(iii) The sampled time optimal feedback systems are said to be uniformly expo- 
nentially stable for all 0 < h < ho, if the constants Mh and a/, > 0 in (ii) 
above are independent of h for 0 < h < h 0 . 

The following result is an important property of the solutions of the Riccati equa- 
tions on the infinite time interval when the optimal feedback systems are exponentially 
stable. The proof can be found in [BW], [DI], [G] for the continuous time problem, 
and in [GR] (Theorem 2.9) for the discrete time problem. 

LEMMA 3.1. Assume that the continuous time control system and the sampled 
time control system with sampling period h are cost functional stabilizable. If the 
corresponding optimal feedback systems are exponentially stable, then TT O0 (-; 0) , and 
n ji,oo(">0) ore the unique bounded solutions of the corresponding Riccati equations 
(3.3) and (3.4) on the infinite time interval. Furthermore, if G and G h are chosen 
such that G > IIoo(*;0) and G h > IL, )OO (&;0) for all t and k, then the solutions of the 
Riccati equations on the finite time interval, Tl tf {t]G) and II M/jJk (Jfe; G h ), satisfy 

<n t/ («;G)^-n co (t;0)^^> jr < <GS{t f ,t)4>,S{t f ,t)<j>> H , 

and 

< G h )4> - n fc)0O (&; G h )<j>, <f> > H < < G h S h (kf ih ,k)4>,S h (k fih ,k)<f> > H , 

respectively, for all t <t f , k < k f>h , and <f> e H. 

Lemma 3.2. Assume that the sampled systems are uniformly cost functional 
stabilizable with the optimal feedback systems uniformly exponentially stable for 0 < 
h < ho- Then, the operators G and G h can be chosen as described in Lemma S.l with 
Gh < C ■ I for some constant C independent of h. As tf tends to infinity, II t (-;G) 
converges to n^-jO) uniformly on any bounded subinterval [a, b] of [0, oo) and the 
convergence of ft M/ik (•;<?*) with k f<h = [t f /h] to n» iOO (-;0) is uniform in h for all 
0 < h < h 0 on any bounded submterval [a, b ] of [0, oo) in the uniform operator norm. 

Proof. We prove only the discrete time assertion. The continuous time case 
is completely analogous, if not simpler. The assumption of uniform cost functional 
stabilizability implies that the operators Gh can be chosen as stated in the theorem. 
Then let M and a be the constants in Definition 3.2-(iii). For a given e > 0 and 
t e [a, 6], we can take t f large enough such that CM 2 exp{-2a(t f - t - h 0 )} < e. Let 

kh — [ t/h ], then (kf ^ — kh)h > tf — t — ho for all 0 < h < Hq. Since II/, * (kh\ Gh) > 
^h,oo(k h ; 0), we have , /M ’~ 

~ n fc>00 (t; 0)||x,(^r) = ||n M/k [k h \ G h ) - n fciOO (A: fc ;0)|| z ,(tf) 

= ,u?P < ( n M M (**;G*) - n kfi0 (k ki 0 ))<t>, 4 > H 

- „r p <G » S n( k fMkh)<j>,Sh(k ft h,kh)4>>„ 

< < e . 
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□ 

As a consequence of the above lemma, we obtain our first convergence result. 

THEOREM 3.2. Let Conditions (Al)-(A4) for the operator families {T h , B h ,Q h , Rh} 
hold on any finite subinterval of [0,oo). Assume further that the continuous time sys- 
tem and the sampled time systems with 0 < h < ho are uniformly cost functional 
stabilizable, and that the optimal closed-loop evolution systems are uniformly expo- 
nentially stable . Then, the Riccati operators IU iOO (t;0) converge strongly to floo^O) 
and the convergence is uniform on any bounded subinterval of [0,oo). 

Proof. Let <j> G H and let (a, 6] be a bounded subinterval of [0, oo). We choose 
an operator G such that G > noo(t;0) and G > rb, iOO (fc;0) for all t € [0, oo) C R, 
k e [0,oo) C Z and 0 < h < h 0 . By Lemma 3.2, t f can be taken large enough such 
that for all kf h = [tf/h], we have 

||n t/ (t; G)4> - n oo (t; 0)<£|| h < | and ||n MM (i; G)<f> - n h)0 oMMk < |, 

for all t e [a, 6] and all 0 < h < h 0 . By Theorem 2.4 of Section 2, we can find h small 
enough such that 

for all t E [a, b]. Therefore, we have 

||nfc,oo(<;O)0-IIoo(<;O)^||H < ||n kl oo(i;0)^-n fc , fcM (t;G)^||jr 

+ ||n MM (t;G)^-n t/ (t;G)0||H + ||n t/ (t;G)^-n oo (t;O)^|| i? < e, 

for all t 6 [a, 6]. □ 

We note that although the exponential stability of the optimal feedback systems is 
only a sufficient condition for the uniqueness of the solutions to the Riccati equations, 
it also implies the stability of the solutions to the Riccati equations under small 
perturbations (see [BW], [DI]) which is important in the context of approximation. 
Consequently the remainder of our discussions here are concerned with conditions 
which guarantee the exponential stability and uniform exponential stability of the 
optimal feedback systems. 

A useful characterization of exponentially stable evolution systems is given in a 
result due to Datko in the continuous time case (see [D] ) and Zabczyk in the discrete 
time case (see [Z]). We state it here in both its continuous and discrete time forms as 
a lemma. 

Lemma 3.3. 

(i) Let T be a strongly continuous evolution system. If there exists constants 
C U C 2 , and u> > 0 such that 

/ OO 

\\T(t,sW„dt < CM\h, 
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for all 4> € H and 0 < s < t < oo, then, we can find constants M and a > 0, 
depending only on C i, Ci, and oj, such that ||T(t, s)||x,(h) < M exp{ — a(t — s)}, 
for all 0 < s < t < oo. 

(it) Let Th be the discrete time evolution system defined by 


Th(i,j) 


/, % = j\ 

n’fclj A h (k), i > j. 


If there exist constants Ci^Uh and C such that 


l|r*(i,j)|U m < and fcOUMMlir < C,,»|W|i, 

i—h 

for all 0 < k < oo and <f> £ H , then, we can find constants Mh and a /, > 0, 
depending only on Ci t h,^h and Cj^, such that for all 0 < j < i < oo 

||r*(t,i)|| L( ff) < M h e~ a ^ h . 


The converse of this lemma is obviously true. By the uniform cost functional 
stabilizability, the solutions of the Riccati equations (3.3) and (3.4), Iloo and IIh i00 are 
uniformly bounded, and therefore, the evolution systems S and S h given by (3.5) and 
(3.6), respectively, axe uniformly exponentially bounded if T and 7\ are uniformly 
exponentially bounded. Moreover, we have 

/ OO 

\\Q{t) 1/2 S{t,s)4>\\ 2 H dt < M\\<j>\\)j 

and 

OO 

k=i 

for some constant M and for all <j> E H. If the operators Q(t) and Qh{k) are uniformly 
strictly coercive (i.e., there exists a constant q > 0, such that Q(t) > ql and Qh{k) > 
ql, for t > 0 and k > 0), we can immediately conclude that S and Sh are uniformly 
exponentially stable. 

A more general case in which the boundedness of the cost functional implies the 
stability of the feedback system, is the case of detectable systems. 

DEFINITION 3.3. (Detectability) 

(i) A continuous time control system is said to be detectable with respect to 
the cost functional (3.1), if there exists a bounded operator-valued function 
V(-) : [0,oo) i — +■ L(H) such that the evolution system Tv, corresponding to 
the perturbation of T by VQ l ^{-), is exponentially stable. 

(ii) A sampled time control system is said to be detectable with respect to the cost 
functional (3.2), if there exists a bounded sequence of operators { (/c) } 

L(H) such that the discrete time evolution system Tv t h given by 

I, i = J, 

U\=)(A^ k ) + kV h (k)Qh{ky /2 ), i > j, 


Tv,h{i,j) = 

is exponentially stable. 
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(Hi) The sampled time systems are said to be uniformly detectable for 0 < h < h 0 , 
if there exist constants C„C,, and a > 0, independent of h such that the 
operator-valued sequences {V»(*)}?.o in (“) satisf y IIU(*0lll.(H) < c i and 

l|2V.a(«,j)lk(H) < C, 0 < j < .' < oo, 

for all sampling rates 0 < h < ho. 

Under appropriate conditions, the detectability of the control systems implies the 
stability of the optimal feedback systems. Indeed, toward this end, we require the 
following boundedness assumption. 

(B) The continuous time evolution system T and the discrete time evolution system 
T h are uniformly exponentially bounded on A(2,oo). That is, there exist 
constants M and u> such that 

||T(t,s)|U ( H) < ||T*(i,j)||x.(ir) < 

for 0 < s < t < oo and 0 < j < i < oo. The operator families B,Q, and 
R and the piecewise constant operator families B h ,Q h , and R h are uniformly 
bounded in norm by a given constant C on the entire interval [0, oo) for all 
sampling rates h > 0. Furthermore, there exists a constant r > 0 such that 
R(t) > rl and R h (t) > rl for all t > 0, and h > 0. 

THEOREM 3.3. Consider a detectable continuous time control system and a 
detectable sampled time system which are both cost functional stabihzable. Assume 
that the evolution systems T, T h are exponentially bounded, and the operator families 
{B,Q,R} and {B h , Qh, Rh} are bounded in norm on the infinite time interval. Then, 
the optimal feedback systems for both systems are exponentially stable. Furthermore, 
suppose that constants C,u,r > 0, and a > 0 can be found such that the following 

conditions are satisfied. . 

(i) The operator families {B, Q, R, IIoo^; 0), V} and {Bh, Qh, Rh, rU,oo( - ) 0)? >*} 

are bounded in norm by C ; 

(ii) For all t > 0, k > 0, R(t) > rl and R h > rl; 

(Hi) The evolution systems T, T^,Tv, and Ty t h satisfy 

||T(t,«)|U{ff) < Ce u l*-\ ||Tv(*, S )||l ( h) < Ce~ a{t - a \ 

and 

l|J»(«',j)||l(») < ||Tv(.',»|U(iO < 

Then there exists constants M and (3 > 0 depending only on the constants C , r, a and 
u such that 

||S(t, s )|U w < Me- 8 "-*', ||Sk(*',j)|U ( H) < 
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Moreover, under Assumption (B), if the sampled systems are uniformly detectable and 
uniformly cost functional stabilizable for 0 < h < h Q} then, the optimal closed-loop 
systems are uniformly exponentially stable for 0 < h < ho. 

Proof. In the case of continuous time system, a proof is given by Da Prato and 
Ichikawa in [DI] . The dependence of the exponential bound for the optimal closed- 
loop system on the constants indicated above is proved in [W]. The arguments for the 
discrete time case are very similar to those used in the continuous time case. Indeed, 
let Sh correspond to the perturbation of 2V,;, by A*, = {A /,(fc) = — Bh{k)Fh(k) + 
i n the sense that 

Sh{t,s)<f> = T v ,h{t,s)(f> + / Ti^f, r) Aj»(r)S* ( 7 , s)<£dr. 

J[s/k\ 

Let us define 

fh{k,i) = -R h {k) ll *F h {k)<t>, and g h {k,i) = Q h {k) 1/2 S h (k,i)(j>, 
for k > 2 - > 0. Then cost functional stabilizability implies that 

HA('>*)lk(*',°°;tf) ^ CWHh, *") ||ij(i,oo;H) < c\\4>\\h. 

The evolution system TV, a is bounded; ||?V i ^(t, J ) || < C exp{— o:(t — Thus we 
obtain 

J\sfh\ 

+ll^( r )IU(H)||g/,(^s)|| ff )d7, 

and by Young’s inequality (see, [A, Theorem 4.30, p.90]), we have 

/ OO 

\\S h (t,s)<i>\\ 2 H dt<K\\<i>\\ 2 H ,ct>eH, 

for some constant K. Applying Lemma 3.3, we obtain the exponential stability of 
Sh • The dependence of the exponential bound for Sh on the indicated constants of 
course follows from the dependence of the constant K on the indicated constants as 
prescribed in the lemma. In this way it is easy to see how under Assumption (B), uni- 
form detectability and cost function stabilizability will imply the uniform exponential 
stability of the closed-loop systems. □ 

Another closely related control theoretic concept is the stabilizability. 

DEFINITION 3.4. (Stabilizability) 

(i) A continuous time system is said to be stabilizable, if there exists a bounded 
operator-valued function Tf(-) : [to, 00 ) >— ► L[H,U ) such that the evolution 
system Tk corresponding to the perturbation of T by BK is exponentially 
stable. 
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(ii) A sampled system is said to be stabilizable, if there exists a bounded sequence 
of operators {K h {k)}? =0 C L(H,U) such that the discrete evolution operator 

T K ,h given by 

f I, * = 

T K<h {i,j) = | n i-i (A fc (*) + hB h {k)K h {k)), i > j, 

is exponentially stable. 

(iii) The sampled time systems for are said to be uniformly stabilizable for 0 < 
h < h 0 if there exist constants Ci,C 2 ,a > 0 independent of the sampling 
period h, such that K /, and Tk,h satisfy 

\\K h {k)\\ L{HiU ) < Ci, \\T Kih {iJ)\\nH) < C 2 e~ a(t 3)h , 
for all 0 < A: < oo, 0 < j < i < oo. 

Using Theorem 3.3 it is easy to verify that cost functional stabilizability and de- 
tectability imply stabilizability (take K = F, K h = F h , for example). Conversely, 
stabilizability clearly implies cost functional stabilizability. Therefore, under the uni- 
form detectability assumption, cost functional stabilizability and stabilizability are 
equivalent. In general, uniform stabilizability and uniform detectability are required 
for the convergence of IT^ i0O to IIoo as h tends toward zero. 

THEOREM 3.4. Let Assumption (B) hold. Suppose further that Conditions 
(A1)-(A4) hold on any bounded subinterval of [ 0, oo). If the continuous time system 
and the sampled time systems are uniformly stabilizable and uniformly detectable, 
then, the unique solution Tl h<00 of the infinite horizon Riccati difference equation (3.4) 
converges to the solution of the infinite horizon Riccati integral equation (3.3) as 
h tends toward zero. The convergence is uniform in time on any bounded subinterval 
of [0, oo). 

Proof. By Theorem 3.3, uniform stabilizability and uniform detectability imply 
exponential stability of the optimal feedback systems (i.e., Definition 3.2), uniformly 

over all sampled systems with 0 < h < h 0 . Therefore, by Theorem 3.2, we obtain the 

, . , □ 

desired convergence. 

Most control systems of interest in engineering practice are stabilizable and de- 
tectable. In fact, in modeling many control systems of practical interest, a realis- 
tic description of the physical system frequently necessitates stabilizability and de- 
tectability of the system model (see, for example, [BKS], [BKSW]). Investigation of 
stabilizability and detectability of particular classes of evolution systems has gener- 
ated several interesting mathematical problems (see, for example, [C], [L]). However, 
in the context of approximation, we usually assume that the original control system 
is stabilizable and detectable. An important issue here is whether or not a given 
time discretization algorithm is capable of preserving, uniformly, these properties, 
and therefore provide discrete time convergent approximations for the optimal feed- 
back operators. In the remainder of this section, we attempt to address this issue for 
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some particular discretization algorithms and indicate some other possible approaches 
to this problem. 

Assume that the control system defined in Equation (2.1) is stabilizable and de- 
tectable with respect to the cost functional (3.1). Thus, there exist bounded operator- 
valued functions K(-) : [f 0 ,oo) »-♦ L(H;U) and F(-) : (t 0 ,oo) L(H) such that the 
evolution systems Tk, Tv, corresponding to the perturbations of T by BK and VQ */ 2 , 
respectively, are exponentially stable. That is, there exist constants M,a > 0 such 
that ||T K -(t,s)|| i (^) < M exj>{-a(t - s)} and ||2V (^, «) ||i(jy) < Mexp{-a(t - s)}, for 
all 0 < s < t < oo. By definition, the evolution operators Tk, and TV satisfy 

(3.7) T K {t, s)4> = T(t, s)<f> + J* T(t, V )B{r,)K{r,)T K {r,,s)<j>dr,, 

(3.8) Tv{t,s)<f) = T{t,s)<t> + j T{t,r])V[ri)Q 1 l 2 {ri)Tv{r),s)<t>dr), 

for all <j> € H and for all 0 < s < t < oo. Consider the zero-order hold discretization 
described in the Section 2. For each k > 0, the operators Ah{k), B h (k) are defined by 

(3-9) A h (k ) = T((k + l)h,kh), 

(3.io) B h (k ) = - / T((k + i)h, n )B(n)dn, 

ft J kk 

with the discrete evolution systems TV*,2V t * then given by 


(3.11) 


» = J, 


ntjOM*) + hB h (k)K(kh)}, i > j , 


(3.12) 


T vA>,j) ( rii=i{>l fc (*) + AK(*/i)Q*(*:) 1/2 }, i>j 


If the discrete time evolution systems TV, a, TV,* are uniformly exponentially stable for 
all 0 < h < ho for some ho > 0, then, these sampled time systems are uniformly 
stabilizable and uniformly detectable. Using (3.7) and (3.8), the evolution systems 
TV, and TV satisfy 


/•(JH-i)h 

T K {ih,jh) = Yl T((k + l)h,kh)+ T{{k + l)h t ri)B(ri)K{ri)TK{vM)dri , 

Je=V . Jkh 


•- 1 r( k + l ) h 1 

Tv{ih,jh) = n T((k + l)h,kh)+ T[(k + l)h,ri)V{n)Q{Ti) l t 2 Tv{Ti,kh)dTi , 

t=, _ Jkh 


for 0 < j < i < oo. Therefore, Tk,h, and TV/, can be considered as perturbations of 
Tk and TV, respectively. In fact, we have 


(3.13) 


(3.14) 


TK,h{i,j) = IJ{Tir((A: + l)h,kh) + h$ h (k)}, 

k=i 

t-i 

Tv,h{i,j) = IJ{^((A: + l)h,kh) + /^(A:)}, 

k =j 
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for 0 < j < i < oo, where 

$ k (lc) = ^ J^ +1)k T (( k + 1 ) /l > ^BMiKikh) - K{r])T v {r),kh)\dr^j , 

and 

V h (k) = ^F(ik/i)g fc (fc) 1/2 -^^ +1)fc T((A: + l)/i,f7)V r (r/)g(r/) 1/2 rv(»7,fc/i)^J » 

for k > 0. Let 0 < u < a and define T Kih {i,j) = exp{w(* - j)h)}T K>h {i, j), 0 < J < 
i < oo and T*(i,s) = exp{w(t - s) }T K {t, 5 ), 0 < s < t < 00 . We define TV and T v ,h 
analogously. It is not difficult to verify that 

||Tjr(t,s)|U(H) < M, ||Tv(t,s)|U(H) < M. 

Multiplying both sides of (3.13) and (3.14) by exp{u;(t — j)h)}, and rewriting these 
equations in a variation of constants form, we obtain 

TK t h{hj) = + h Tjcji h, ( k I l)/i)e j), 

A :=i 

Tv,h{iJ) = fv{ih,jh) + hY t Tv{ih,{k + 1 )h)e uh * h {k)f v ,h{k,j). 

k-j 

If there exists a constant h 0 > 0 such that for all h < h 0 , exp{u>/i}||$>,(A;) \\l(H) < w/2M 
and exp{w/i}||^h(A;)||x,(H) < u/2M, then, 

»-i u 

\\f Kih {i,j)\\ L {H) < M + h^, M ■ j)\\l(h), 

k=j 

\\Tv,h[i, j)||i(/r) < M + h y~) M • -^j^\\Tv,h{k, j)\\l{H) ■ 

k=j 

The discrete Gronwall inequality then yields 

Therefore, 7V/,, and TV , h are uniformly exponentially stable for all 0 < h < Hq. 

It is not difficult to see that for each k > 0, $ h (fc) and V h {k) converge strongly 
to zero as h tends toward zero. We can obtain convergence in norm if the rank of the 
operator valued functions $>,(/:) and ^(A;) is finite. 

DEFINITION 3.5. (Finite rank operator-valued function) Let X and Y be Hilbert 
spaces with inner products < •,• >x and < •,• >y, respectively. An operator-valued 
function W (•) : [0, 00 ) 1 — ► L(X,Y) is said to be continuous and to have finite rank, if 
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there exist continuous vector-valued functions /*(•) : [0, oo) >— ► X and g k (-) : [0, oo) i— ► 
Y , k = 1, • • • , n with n < oo, such that for all x G X, 

«'(*)*= E </*(«).* >r «»(*)• 

fc=l- 

We define the following condition. 

(F) (Finite Rank Stabilizability and Detectability Condition) There exist finite rank 
continuous operator-valued functions K(-), V'(-) such that the perturbed evo- 
lution systems Tk and Ty are exponentially stable. 

LEMMA 3 . 4 . Suppose that Conditions (A 1 )-(A 4 ) hold. If the finite rank con- 
dition (F) is satisfied, then on any finite subinterval of [ 0 , oo), the operator-valued 
functions $/,, &nd 'Fj, constructed from and 'F/, in the usual manner, converge 
uniformly to zero in the uniform operator norm as h tends toward zero. 

Proof. We consider 'I'* only, the argument for is analogous. Using the finite 
rank condition, we write 


y (00 — < fk{t),<i> >n g k {t), 

k=l 

with fk and g k continuous for k = 1, • • • , n. It follows that 

V(ih)Q h (.y'V = E < >h «»(••*), 

A = 1 


for i > 0, and 


r(t,r7)V r (r/)g(77) 1/2 T K (r7,6)0 = ^ C TVfa, ^s) ^ Q (77) 1/2 /jk (77) , <f> > H T{t,r))g k {n), 

k= 1 

for 0 <s<T]<t<oo. Therefore, we have 

*400 = F([t/fc]A)Q*(«) l /V 

~l J mh r((|t/fc| + 1)4, r,)V( V )Q(n)'l'T v (,, |1/A]4)M? 

1 JL /•([«/*] + !)* r 

= T E L < <W‘) 1/ 7»([‘/*W,* >* S,([(/fc]/>) 

^ jfc=l •'I'/ "J" 

- < iv(ij,|«/a]a) > <J(i>) 1 '7,(.j),* >„ r((|t/*| + i)M)j*0?)} <*)• 

By adding and subtracting the term < Ty (rj, [t/h]h)*Q(r}y / 2 f k (i]), <f> > H g k ([t/h]h ) 
under each of the above integral signs, and using the Schwartz inequality, we obtain 
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the following estimate 




h J{t/h]h 

+Vh(t,n)l|rv(n,[i/^]M*QW 1/2 /k(n)lk}||^llH^ 


-tl 

h r< Jb 


((*/*!+ 1)' 1 


h j 


Uh(^^)ll < A|U dT ? 


where 


Uh(t,rj) 


||Q h (f) 1/2 / fc ((i//t]/i) - Tv(r?, [f/fe]/0*Qfa) 1/2 /kfa)ll*. 
||g([t/fcj&) - T(([t/fc] + l)h,rj)g{v)\\H, and 


Since the functions /. and Sk are continuous on any bounded subinterval [ O 6M[0^ ), 
and for any e > 0, there exists 6 > 0 such that \\f k (t) - AOOIU < « and 
j, t ( s )|| H < e for all t, s 6 [a, 6] with |t-a| < « and fc = 1, • • • , »■ Then ’ 1 e 
of the operator families T t Tr,V t Q k ,Q and the uniform strong convergence of Q, to 

Q implies that for any bounded subinterval [a, 6] of [0,oo), ^^^fmctions 

e > 0, we can find h 0 > 0 such that for all 0 < h < h 0 and | i the f unct °n 

u h (t,rj) < e for V € [M + h\ and f € [a, b\. Consequently ||**MIU(J0 ^ for q 

‘ £ w! can extend the uniform convergence on finite time inmrvals to «nta convey 
gence on the infinite time interval by assuming certain periodicity (m 
invariance) of the evolution system T and the operator-valued functions 
and V. In fact, the periodicity assumption implies that $/,, h are a so peri 

functions of time. 

THEOREM 3.5. Assume that the evolution system T and the operator val ^ 
functions B Q R are strongly continuous and periodic with the same period, 6 Sup- 
‘ZTZliat the periodicity o, Q is preserved by Q h for the sompfed t.me spste^ 
If the finite rank condition (F) holds for some 0-penodic functions K and V the 
the discretization defined in (3.9) and (3.10) jenerutes aniformly 

formly detectable sampled control systems for samphn, pertods h mth 0 < h < he for 
some constant h 0 > 0. 

The periodicity assumption is trivially satisfied in a large number of Practical ex 
amples, in particular, it is satisfied for all time invariant systems However he 
rank assumption says, in essence, that only a fin, te number of modes of the state 
vector are unstable in the absence of control. Indeed, ,n the case of evolution system 
corresponding to a hyperbolic partial differential equation, there exists examples in 
which if the finite rank condition is not satisfied, all sampled systems are not stab - 
Usable even though the continuous time control system is stabilise. For parabolic, 
compact, evolution systems, the spectral properties of the evolution system provide 
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valuable additional structure. In this case, an approach which does not require the 
finite rank condition, similar to the one used in [R], can be applied. The results using 
t is type of argument will be reported on elsewhere. However, even in the case of 
parabolic, compact, evolution systems, since the unstable spectrum consists of only 
a nite number of isolated points, it would be interesting to know whether these sys- 
tems can be stabilized via finite rank feedback. If the answer is affirmative, then the 
arguments presented here may not be as restrictive as they seem. For other discretiza- 
tion schemes, the uniform stabilizability and uniform detectability of the generated 
sampled systems remains, in most cases, an open question. 


4. Examples and Numerical Results. In this section we present and briefly 
discuss some of our numerical findings which serve to illustrate our convergence results 
m t e context of a variety of distributed parameter control systems. In particular, 
we consider the infinite horizon optimal control or regulation of a heat or diffusion 
equation, a delay or hereditary system, and a flexible structure in the form of a 
cantilevered Voigt-Kelvin viscoelastic beam with tip mass. 

In all of the examples to follow, we consider time invariant systems only, and 
o tain the discrete or sampled time operators from the corresponding continuous 
time operators via T h - T(h ) ), B h = h~' f 0 h T(t)Bdt, Q h = Q, and R h = R, for 

* > 0 (i.e via zero-order hold sampling). In order to solve the resulting infinite 
dimensional continuous and discrete time LQR problems, we introduced some form 
ot state discretization (i.e. either modal or spline based Ritz-Galerkin techniques) 
which were known to yield convergence in the closed-loop problem. By choosing the 
state discretization sufficiently fine, we could assume that we obtained a reasonably 

accurate finite dimensional approximation to the solution of the infinite dimensional 
LC^K problems. 


The resulting finite dimensional continuous and discrete time LQR problems 
(more precisely, the matrix algebraic Riccati equations) were solved using either eigen- 
vector (in the continuous time case, also known as Potter’s method, see [KS]) or Schur 
vector (for the discrete time problems, see [PLS]) decomposition of the Hamiltonian 
matrix. All computations for the first two examples were carried out on an IBM PC 

AT. The flexible structure problem was solved on an IBM3090, although it too could 
have been solved on a personal computer. 

In each of the examples below, the control systems are time invariant and the 
control space U is finite dimensional. In fact, U = R. Thus, the optimal feedback 
gains, F and F h , are elements in L(H,R). That is, they are bounded linear functionals 
on B. Consequently, they admit representors, respectively / and f h , in H with F<p =< 
f,<P >h and F k <p =< f h , <p > H , for <p e H. The elements / and f h in H are referred 
to as the optimal continuous or discrete time functional feedback control gains The 
finite dimensionality of the control space U also implies the uniform stabilizability 
o he sampled systems when the continuous time systems are stabilizable (recall 
heorem 3.5). Our convergence result implies that lim^ 0+ F h <p = F<p for <p e H. 
,° 6 \ at w ^ en U is finite dimensional, this is equivalent to lim fc _ 0 + F h = F in 
the uniform norm topology on L(H, U) and linw f h = /in H . It is this latter 
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convergence of the functional gains which we shall exhibit in our plots below. 
Example 4.1. We consider the scalar or one dimensional heat or diffusion control 
system 

J^z(f,77) = a-g^x(t, rj) + &X[«i.«a] ( T ?) t4 (*)» 0 < V < M > 0> 

with the Dirichlet boundary conditions 

x(t, 0) = x(t, l) = 0, t> 0, 

at 77 = 0 and rj ~ 1 where a > 0, b G 0 < £i < €2 ^ 1? an< ^ Xs denotes the 
characteristic function on the set 5. We take the performance index to be 

J( u ) = j 0 (f 0 *?)**? + ru(t) 2 }dt 

with <7 > 0 and r > 0. 

In this case we have H = L 2 (0, 1), U = R, A : Dom(A) C H H given by 

A<p = aD 2 tp for <p 6 Dom(A) = H 2 ( 0, 1) P| ^o(0, 1), 

B 6 L{R,H ) given by [Bv){t]) = 6x[ £l>£3 ](r?)v, 0 < ry < 1 , v e R, Q 6 L(H) given 
by Q = ql, and R € L[U) given by R = rl , where / denotes the identity map on 
R. We note that {T(t) : t > 0}, the semigroup of bounded linear operators on H 
with infinitesimal generator A, is parabolic and uniformly exponentially stable. Thus 
the continuous time pairs, {A, B} and { Q,A } are trivially stabilizable and detectable 
and the discrete time pairs, and {Q fc ,T\} are uniformly stabilizable and 

detectable as well. 

Setting a = 0.1, b — 1.0, q = 1.0, r = 1.0, e\ = 0.21, and ti — 0.275, we 
obtained the plot of the functional gains / and f h in L 2 (0,l), for various values of 
h > 0, given in Figures 4.1 and 4.2. Those in Figure 4.1 were obtained via a modal 
(i.e. sin(A:7rz), A: = 1,2,---,JV) state discretization with N = 20 modal elements. 
For the gains in Figure 4.2, we used linear B-spline elements (i.e. “hat” functions) 
defined with respect to a uniform partition of [0, l] into N = 20 subintervals of equal 
length. Convergence of these state approximations and the corresponding closed-loop 
solutions to the control problem is well known (see, for example, [G], [GR]). 
Example 4.2. In this example, we consider the scalar, single input hereditary control 
system 

(4.1) x(f) = a 0 x(t) + a x x(t - l) + bu(t) 

where a 0 ,a u b G R. We take the performance index to be 

r oo 

J(u) = / {qx 2 {t) +ru 2 (t)}dt 

Jo 


with q > 0 and r > 0. 
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Table 1 

Head, gams for hereditary system. 


Sampling period h 

Head gain / u 

10' 1 

3.76185 

10' 2 

4.35007 

10' 3 

4.41577 

10' 4 

4.42241 

10' 5 

4.42308 

10' 6 

4.42314 

Continuous time 

4.42315 


The abstract Hilbert space formulation for linear hereditary control system* * 

well known (see, for example, [BB]). We let H = RxL 2 (-l,0),U - « and set 

A • DomU) Cl?-fftobe A(r,,<p) = («of» + for G f° m(A) " 

{(f,« 'me = *(0)}. The operator A is the infin.tee.mal generator 

of the Co-semigroup of bounded linear operators on If, {T[t) . }, g 

TltHn v) ( xit) l) where x is the solution to (4.1) with u.Oand corresponding 

T[t) t V 'r . - W \h- x < _ = (9) _! < o < o, and x t e L,(-1,0) is the 

to the initial data x(0) T), x( ) *’ . i <r fl < fi We let 

past history of x from t back to t - 1. That is x t (0) - x(^ ), - = , 

B € L{R,H) Q e L(H), and R € L(tf) be given by Bv - (6v,0), Q(»f,V>) I? 7 ?- )> 

a " d To" solve’ bot^th'r continuous and discrete time LQR problems we employe^ 
piecewise constant/, inear spiine hybndjinite el-en^scheme 

Kappel in [IK]. Setting a 0 — ai — b q ’ , ,1 p v /" 1 0) 

level in the Ito-Ka PP el scheme taken to be If = 20, we obtained the fix L,( 1,0) 
functional gains, / = (/»,/*) and f h = (A°,/i) for venous values of £> 0 t*bulattd 
and plotted in Table 1 and Figure 4.3 below. We note that for this . choice of t 
parameters a„ and o„ the open loop system has an eigenvalue with positive 
Consequently the system (4.1) is open-loop unstable. It is not difficult to argne ha 
the pairs {.4,B} and {<?,A} are respectively stabilizable and detectable. Also, since 
the operators B and Q are of finite rank, there exists ho > » such that for al I samp 4ng 
periods h < ho, the sampled control systems are uniformly stabilizable and detectable 

Example 4.3. We consider the control of the small amplitude transverse vibration of 
a cantilevered Voigt-Kelvin viscoelastic beam with tip-mass. The rel, svant dynami 
are described by the hybrid system of ordinary and partial differential equatio 

+ = 6 (0 ’ X) ’ 

d 2 . . - d 4 


m dt 2 x( < t ' 1 } cI dr}*dt 


z(t,l)-E/|^r(t,l) = M‘). 
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Fig. 4.3. 
(t) Continuou 


Functional gain, for delay equation, (a) h=0.1, (b) h=0.01, (c) h.=0.001 


(d) h=0.0001, 



for t > 0, the essential (or stable) boundary conditions at r) — 0 

Q 

x(£,0) = 0, —x{t, 0)=0, £ > 0, 


and the natural (or unstable) boundary condition at rj = 1, 


cl 


d 3 


dr] 2 dt 


d 2 , x 

x{t, 1) - EI—x(t, 1) = 0, 


t > 0. 


In the above equations p > 0 is the linear mass density of the beam, I > 0 is the 
beam’s cross sectional moment of inertia, c > 0 is the viscosity coefficient, E > 0 is 
Young’s modulus, m > 0 is the mass of the tip mass, and b G R is a constant. 

We take an energy based performance index: 


+ jf + 

Once again the abstract Hilbert space formulation of this problem is standard. We 
let H = H 2 L { 0, 1) x R x L 2 (0, 1 ) where # 2 ( 0 , 1 ) = {y> € # 2 ( 0 , 1) : p(0) = Dtp{ 0 ) = 0}, 
and endow H with the energy inner product 




El C D 2 <p x D 2 v 2 
J o 

+mrh r/2 + p 


f 

./O 


The operator A : Dom(A) C H i— ► // is given by A(<p>,77,V>) = (4>,cID 3 ip( l) + 
EID 3 ip{l),-cID*rp - EID 4 <p) for (^,r?,0) G Dom(A) = {(v?,h,0) € H : 0 € 
#£(0,1), rj = iJj(1),cID 2 tP + EID 2 <p € # 2 (0, 1), c/Z? 2 ^!) + £J£>V(l) =0}- We take 
U = R and define 5 € £(#,#) by Hu = (0>,0). We let Q G L{H) and R G L[U) 
be given by Q = (l/2 )/# and R = r/^, where /#, and Iu denote, respectively, the 
identity operators on H and U. 

It can be shown (see [G A] ) that A is the infinitesimal generator of a uniformly ex- 
ponentially stable analytic semigroup. Thus once again stabilizability and detectabil- 
ity for the continuous time problems trivially follows as does the uniform stabilizability 
and detectability for the discrete time problems. 

We employed a standard cubic spline based Ritz-Galerkin finite element scheme to 
approximate or finite dimensionalize the continuous and discrete time LQR problems 
(see [GA], [GR]). Setting p = 0.1,#/ = 1.3333 x 10" 4 ,c/ = 1.3333 x 10- 7 ,m = 1,6 = 
l,q = 1, and r = 1 and with N — 9 cubic spline elements, we obtained the functional 
gains / = (/°, f 1 , f 2 ),fh — (/£, fhifh) € H exhibited in Table 2 and Figure 4.4 below. 
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Table 2 

Tip gains for beam equation. 


Sampling period h 

Tip Gain, f l 

1.000 

0.12181 

0.500 

0.12003 

0.010 

0.11798 

0.005 

0.11796 

0.001 

0.11794 

Continuous time 

0.11793 


FlG. 4.4. Functional gams for beam equation, (a) Displacement, we plot D 2 f° to exhibit the H~ 
convergence; (b) Velocity. 



(a) (b) 
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5. Summary and Concluding Remarks. We have investigated and estab- 
lished the convergence of solutions to discrete or sampled time linear quadratic regu- 
lator problems and the associated Riccati equations for infinite dimensional systems 
to the solutions to the corresponding continuous time problem and associated Riccati 
equation, as the length of the sampling interval tends toward zero. We have con- 
sidered both the finite and infinite time horizon problems and carried out numerical 
studies involving a variety of distributed parameter control systems in order to ob- 
serve how well our theoretical results predict what actually takes place in practice. In 
the context of the finite time horizon problem, the assumption of strong continuity on 
the operators which define the control system and performance index, together with a 
stability and consistency hypothesis on the sampling scheme, are sufficient to establish 
the strong convergence of the Riccati operators, feedback gains, optimal control laws, 
and optimal trajectories, with some degree of uniformity in time over the compact 
interval of interest. For the infinite time horizon problem, we require the additional 
assumption of stabilizability and detectability, uniformly with respect to the length 
of the sampling interval. We have shown that this condition can be verified when 
zero-order hold sampling is employed and the continuous time system is stabilizable 
and detectable by finite rank feedback. We also have a result for parabolic systems, 
but this will be reported on elsewhere. 

Several interesting open questions related to the results we have presented here 
remain open. For example the inter-relation between stabilizability /detectability for 
the continuous and sampled time systems in a more general setting and under more 
general sampling schemes (A-Stable Pade, for example) requires further study. Also, 
convergence under simultaneous and independent state (space) discretization (i.e. 
finite difference or finite element approximation) and temporal sampling should be 
investigated. It would not be difficult to extend the results we presented here to 
handle certain “coupled” state and time discretizations. Finally, a study similar to 
the present one could be carried out for the LQG estimator and compensator problems. 
We have not as of yet looked at these problems, but suspect that similar results to 
those given above could be obtained. 


REFERENCES 


[A] 

[BB] 

[BKS] 

[BKSW] 

[BK] 

[BW] 


Adams, R.A., Sobolev Spaces , Academic Press, Inc., 1975. 

Banks, H. T. and J. A. Burns, Hereditary control problems: Numerical methods based on 
averaging approximations , SIAM J. Control and Opt., 16 (1978), pp. 169-208. 

Banks, H.T., S.L. Keeling, and R.J. Silcox, Optimal control techniques for active noise 
suppression , Proc. 27th IEEE Conf. on Dec. and Control, Austin, TX, Dec. 7-9, 1988, 

pp. 2006 - 2011 . 

Banks, H.T., S.L. Keeling, R.J. Silcox, and C. Wang, Linear quadratic tracking problems in 
Hilbert Space: Application to optimal active noise suppression , IFAC Proc. 5th Symp. on 
Control of Distr. Parameter Systems, June 26-29, 1989, Perpignan, France, pp. 17-22. 
Banks, H.T., and K. Kunisch, The linear regulator problem for parabolic systems , SIAM J. 
Control and Opt., 22(1984), pp.684-698. 

Banks, H.T. and C. Wang, Optimal feedback control of infinite -dimensional parabolic evolu- 


37 



[D] 

(C] 

[DI] 

[Gj 

[GA) 

[GR] 

[HH] 

|H] 

[HK] 

[IK] 

|K] 

IKS] 

[L] 

[LCB] 

[Le] 

[PLS] 

[R] 

[RM] 

[RW] 

[W] 

|Z] 


tion systems: approximation techniques , SIAM J. Control and Opt., 27 (1989), pp.1182- 
1219. 

Datko, R., Uniform asymptotic stability of evolution processes in a Banach space, SIAM J. 
Math. Anal., 3 (1972), pp.428-445. 

Chen, G., Energy decay estimates and exact boundary value controllability for wave equation 
in a bounded domain , J. Math. Pures Appl., 58(1979), pp. 249-273. 

Da Prato, G and A. Ichikawa, Quadratic control for linear time varying systems , SIAM J. 
Control and Opt., to appear. 

Gibson, J. S., The Riccati integral equations for optimal control problems on Hilbert spaces , 
SIAM J. Control and Opt., 17 (1979), pp. 537-565. 

Gibson, J. S. and A. Adamian, Approximation theory for LQG optima/ control of flexible 
structures , ICASE Report No. 88-48, Institute for Computer Applications in Science and 
Engineering, Hampton, VA, 1988, and SIAM J. Control and Opt., to appear. 

Gibson, J. S. and I. G. Rosen, Numerical approximation for the infinite dimensional discrete 
time optimal linear quadratic regulator problem, SIAM J. Control and Opt., 26 (1988), 
pp, 428-451. 

Hager, W. W. and L. L. Horowitz, Convergence and stability properties of the discrete Riccati 
Operator equation and the associated optimal control and filtering problems, SIAM J. 
Control and Opt., 14 (1976), pp. 295-312. 

Hautus, M. L. J., Controllability and stabilizability of sampled systems, IEEE Trans. Auto. 
Control, AC-17 (1972), pp. 528-531. 

Hersh, R. and T. Kato, High-accuracy stable difference schemes for well-posed initial-value 
problems , SIAM J. Numer. Anal. 16 (1979), pp. 670-682. 

Ito, K. and F. Kappel, A uniformly differentiable approximation scAeme for delay systems 
using splines, Appl. Math and Opt., submitted. 

T. Kato, Perturbation Theory for Linear Operators, Second Edition, Springer- Verlag, Berlin, 
1976. 

Kwakernaak, H. and R. Sivan, Linear Optimal Control Systems, Wiley- Interscience, New 
York, 1972. 

Lagnese, J., Decay of solution of wave equation in a bounded region with boundary dissipa- 
tion, J. Diff. Equations, 50(1983), pp. 163-182. 

Lee, K. Y., S. Chow, and R. O. Barr, On the control of discrete-time distributed parameter 
systems, SIAM J. Control and Opt., 10 (1972), pp. 361-376. 

Lewis, F. L., Optimal Estimation with an Introduction to Stochastic Control Theory, Wiley- 
Interscience, New York, 1986, 

Pappas, T., A J. Laub, and N. R. Sandell, Jr., On the numerical solution of the discrete-time 
algebraic Riccati equation , IEEE Trans, auto. Control, AC-25 (1980), pp. 631-641. 

Rosen, I.G., Optimal discrete-time LQR problems for parabolic systems with unbounded 
input- approximation and convergence , Control Theory and Advanced Technology, 
5(1989), pp. 277-300. 

Richtmyer, R. D. and K. W. Morton, Difference Methods for Initial Value Problems, Wiley- 
Interscience, New York, 1967. 

Rosen, I. G. and C. Wang, Preservation of stabilizability and detectability under sampling in 
para6o/tc distributed parameter systems , Proceedings 29th IEEE Conference on Decision 
and Control, Dec. 5-7,1990, Honolulu, Hawaii, submitted. 

Wang, C., Approximation methods for linear quadratic regulator problems with nonau- 
tonomous periodic parabolic systems , Ph.D Thesis, Brown University, May, 1988. 

Zabczyk, J., Remarks on the control of discrete-time distributed parameter systems, SIAM 
J. Control and Opt., 12 (1974), pp. 721-735. 


Appendix A. 

(i) Continuity of P^: Let <f> G U. Then since Ph<j> is constant on each of the 
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intervals Ij = [ jh , (j + l)/i), we have 

iijwiil = 

= e h \\ h ~ l j <^( 5 ) d5 ir 

J=0 JI i 

< E h - 1 

;=0 

= f'-" y(s)\\‘ds < mi 

Jo 

(ii) Convergence of P^: First, let us consider a continuous function <f> G U. Then, for 

all t G [0,t/), there exists fro(t) > 0 such that for all h < h 0 (t), we have t G 

That is, for all t € [0, tj\, 

(P h <t>)(t) = h - 1 J^{s)ds 

for h small enough, where I is an interval of length h containing t. The Bochner 
integral is equal to the Rieman integral for continuous functions, and therefore, for 
all t 6 [0, tf), it follows that 

£?+ ( p ^)(*) = ^(0- 

Now consider an extrapolation operator, E /, : Uh >-+ U, defined for all u € Uh 

as: E h u(t) = u(t), for t G [0, t /,/»], Ehu(t) = 0 for t G (tf ih ,t f ]. It is evident that 

(E h P h <t>){t) converges to <f>(t) as h tends to 0 for all t G [0, t/>). Since 

f f ||(£ h P^)(s)|| 2 ds = ||Pfc^|| Uk , 

JO 

from (i), we conclude, via the dominated convergence theorem, that 

«m ||JW|k = ll^llu- 

Then from the uniform boundedness of P/, and the density of the continuous functions 
in U, we obtain 

lim ||PW|| Ul = ||0||u 

h — ►O'" 

holds for all <f> G U. 

(iii) For all u e Uh we have E h u G U, PhEkU = u, and ||P fc P h u||^ fc = ||^u||u. 

Appendix B. 

We shall show that 

lim \\Bh,3<f> ~ PhB a <t>\\u h = 0, 

h-> 0 + 


f \\m\ 


39 



uniformly in s for s G [0, £/], and <j> £ H. Analogous arguments can be used to show 
that 


lim \\Z h>t P h u - Ph%,u\\u. = 0 

A— o+ 


and that 


J|j“ || 8* h}S PhU - B*u\\„ = 0 

uniformly in s for s £ [0, t /] , and each u £ U. 

From the definitions of 8, and B, given in (2.2) and (2.11) respectively, it is easy 
to observe the following for any <f> £ H. 

(i) ||(B a <£)(t)||y is continuous with respect to s for all t > s. Therefore, from the 
uniform boundedness of ||(B,<^)(<)||i 7 for all 0 < s < t < tf, we conclude that 
B 3 (j> is a continuous function of s in the U norm. 

(ii) By condition (A1)-(A4), it is easy to see that 

$(t,s) = \\B, ih (p{t) - B a <t>{t)\\u 

converges to zero, uniformly in A(2,f/). By the umiorm boundedness of 
HBfcXOIIt, for all h, we conclude that 

ns,,*'# - mi 5. = f , ii + £'* ii s.,*#w - B.mwiit 

J[s/h\h J[3/h\h 

converges to zero, uniformly in s. 

Since we have 

\\8k, 3 4> - Ph&i^UK < II ~ B,(f>\\u h + \\{P h - I)B a <j>\\u h , 

the strong continuity of 3,, and the strong convergence of P /, to the identity 
on U , yield the desired uniform convergence result. 
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